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Attainment of Cognitive Objectives 



Introduction 

The Appalachia Preschool Test (APT) was developed by AEL to supplement 
the standardized instruments which were being used to measure program perfor- 
mance, The APT consists of two principal subtests, an introductory interview, 
and a section measuring logical reasoning and reading skill achievement. 

It should be noted that the APT measures only cognitive program objec- 
tives of a curriculum specific nature and concentrates primarily on those which 
are amenable to multiple choice or sentence completion format, That is, it 
is derived directly from program objectives and has not been subjected to the 
usual item analysis techniques for normed tests. Since the test is not de- 
signed to discriminate between individual children but rather to measure 
achievement by treatment groups of specific program objectives, the statisti- 
cal techniques cited would be inappropriate for program evaluation. However, 
psychometric evaluation of the instrument is under way to provide a basis 
for its use as a normed, standardized measure. 

The preliminary interview of the APT consists of a series of 11 questions 
which serve primarily to introduce the testing situation and to help establish 
rapport between the child and the tester. 

" . - ; •- * - f - 

Part 1 of the APT- has been incorporated into the interview and is no 
longer administered separately. Part 2 is a 61-item, multiple-choice test, 
based primari ly on program ob j ectives from the first year’s curriculum. 

Parts 3 and 4 were Piaget i.an conservation tasks which are no longer empha- 
sized in the curriculum and therefore are no longer included in the APT. 

Part 5 of the APT contains 18 items which measure logical reasoning, 
sensory discrimination and labeling, and letter recognition. Part 5 was 







intended to supplement Parts 2 and 6, in that it reflected secondary program 
objectives not included in the remainder of the test battery. Part 6 is 
similar to Part 2 in format and is based on objectives taught during the 
second and third year of the program. 



During the program year which extended from September, 1970 , until June 
1971 , a number of cognitive objectives from the Hooper-Marshall study ( 1968 ) 
were selected as primary objectives of the television program. Around the 
Bend, and the related activities of the home visitors and mobile classroom 
teacher. 

To measure the attainment of those objectives as well as objectives 
taught during previous years. Parts 2, 5, and 6 were administered in June 
and September, 1970 , and June, 1971 , to a sample of approximately 300 child- 
ren in three treatment groups and to a control group of approximately 120 
children. A description of the sampling procedures involved, as well as a 
description of each group, is included in the introduction to the total 
evaluation. 



Although the APT interview was not primarily intended as a measure of 
program effectiveness, an analysis of the data will be included in this 
report since treatment effects were apparent in the results. It includes 
items such as "What is your name?," and "Show me your nose?” Table 14,1 
identifies APT interview pretest and post-test mean scores, standard devia- 
tions, and sample sizes for each treatment group by age and sex. Overall 
group means also are presented in Table 14.1, showing the scores collapsed 
across sex and age groups. 

Overall post-test mean scores on the APT interview, adjusted for 
chronological age and PPVT post-test raw score, are shown graphically in 
Figure 14,1 
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Figure 14. 1 



APT Interview Group Means Adjusted for PPVT Post- test Raw Score and 




Pre and Post- test Means and Standard Deviations on APT Interview 

by Age, Treatment, and Sex 
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The analysis of covariance table is shown in Table 14.2. 



Table 14.2 

Analysis of Covariance Table--APT Interview 
Post- test Raw Scores 



Source 


9* 

t r 


d.f. 


Mean Square 


F 


P 


Trt. 


.030 


3 


9.79 


4.15 


<.001 


Sex 


.012 


1 


11.49 


4.87 


<.005 


Trt. by Sex 


.002 


3 


0.87 


0.37 




Covs . 




2 


11518.30 


4879,22 




Cov. 1 




1 


302.87 


128.30 




Cov, 2 




1 


133.97 


56.75 




Error 


.954 


387 


2.36 







*This n 2 indicates the proportion of variance allocated among the 
four sources listed . 



The ANCOVA shows both significant treatment effects (p<,001) and sex 
effects (p<.005). A Dunnett's post hoc comparison that the contrast between 
the TV-HV-MC group against the TV only and control groups contributed signi- 
ficantly to the overall treatment effect. The higher scores for the group 
visiting the mobile facility indicate that the objectives on the APT inter- 
view may have been emphasized on the van. Alternatively, the children may 
have been more at ease with the tester after being exposed to the mobile 
classroom, and thus more able to respond to the items on the interview. 
Technical Report No. 18, which deals with social skills development, indicates 
that children on the van initiated more constructive behavior, which substan- 
tiates the former hypothesis. 

Table 14.3 shows mean scores on the APT interview by sex for the three 
treatment groups and one control group. As is evident, neither sex consis- 
tently outscored the other. The tendency for males in the TV-HV-MC group 
to outscore their female counterparts occurs frequently across the instru- 
ments used to evaluate the program effects. This may reflect sampling 
error not removed by covariance or a differential degree of learning taking 
place on the mobile classroom. 



Table 14.3 

APT Interview Mean Scores by Sex and Treatment Group 




Control 



Males 



8.29 


y - 8.07 


ft- 7.38 

..:K -v. 


7.48 


8.13 


8.18 




7.70 



APT Part 2 



Part 2 of the APT is a multiple choice subtest of 61 items developed 
during the first year's progrElmming (1968-69). It consists of a variety 
of items, with the following numbers of items comprising the total: vocab- 

ulary (noun designation) - 15 questions; relational terms - 14 questions; 

letter and number recognition - 6 questions; mathematical sets - 4 ques- 
tions; geometric shape - 4 questions; and beginning and ending sounds - 8 
questions; the balance of the test is devoted to body part identification, 
calendar dates, and time related terms. 

Table 14,4 indicates the treatment group by age and sex cells for APT 
Part 2. The overall means are also presented in Table 14.4, showing the 
scores collapsed across sex and age groups. 

The overall post-test means from the analysis of covariance adjusted 
for chronological age and PPVT raw score are presented graphically in 
Figure 14.2, 



50+ 




TV-HV-MC TV-HV TV Only Control 

40.07 38.60 34.35 + 29.24 



Figure 14.2 

Overall Means Adjusted for PPVT Post- test Raw Score and 
Chronological Age for APT Part 2 









The ANCOVA table for Part 2 of the. APT is presented in Table 14.5 



Table 14,5 

Analysis of Covariance Table for APT Ps*t 2 
Post- test Raw Score 



Source 


n 2 


d.f. 


Mean Square 


F 


P 


Trt , 


.11 


3 


908.54 


16.50 


<,005 


Sex 


.02 


1 


377.14 


6.85 


<.005 


Trt. by Sex 


.00 


3 


21.77 


0.40 




Covs , 




2 


253138.35 


4597,62 




Cov, 1 




1 


1159,94 


21.07 




Cov. 2 




1 


10334,26 


187,70 




Error 


.87 


387 


55.06 









The treatment effect which is evident from the ANCOVA table is clari- 
fied by the results of a Dunnett's post hoc analysis. The TV-HV- MC and 
TV-HV group shared significantly higher scores than TV only and control 
groups but did not differ from each other. The TV only group also scored 
significantly above the control group. These findings point to the con- 
clusion that the television program made a contribution to the overall 
cognitive growth of the child, and that the home visitor added to this 
growth. The mobile facility did not appear to be contributing to this 
area of learning, however. 

The significant sex effect apparent in the covariance analysis is 
due to a generally higher score for females in the 3- and 4-year- old cate- 
gories across all treatments, although 5-year-old males in the TV-HV- MC 
group outscored their female counterparts on this section of the APT and 
other measures. It may well be that the younger girls became more respon- 
sive to learning situations as they progressed through the program, while 
their exposure to the mobile facility enabled boys to continue their in- 
volvement in the program's objectives. 

Table 14,6 indicates mean gain scores on Part 2 of the APT by treat- 
ment, sex and age. 

Table 14.6 

Mean Gain Scores, APT Part 2, by Age and Treatment 



TV-HV-MC 


TV-HV 


TV 


Control 


% 


M 


6.83 


6.86 


1.57 


1.33 


a 


F 


11.75 


4.71 


16.00 1 


1.29 


A 


M 


20,40 


11.14 


13.88 


Ol 


*T 


F 


14,86 


11.11 


12 . 00 1 


* 25 


c 


M 


12.00 


10.82 


14700 


8.00 




“ F 


9775 


13.93 


i 9.57 | 


| 7.00 



An analysis of covariance using age and PPVT raw score as covariates, 
indicated the presence of a treatment effect (p<.005) in these gain scores. 

No sex effect was present. Generally, the TV-HV- MC group showed the highest 
overal 1 gains in the 3- and 4- year- old groups , while among 5-year-olds , gains 
tended to be uniform across the three treatments. 

i apt Part 5 

Part 5 of thei APT was not derived from a separate listing pf program 
objectives but instead was elicited from the) curriculum materials' team ;asYii/ ; .. 





a supplement to Part 2 of the APT, Part 5 of the APT is a verbally oriented 
measure of cause and effect relations and reading skills. 1 

Table 14.7 indicates pretest and post-test means, standard deviations, 
and sample size for each treatment group by age and sex cells for APT Part 
5, The overall means also are presented in Table 14.6, showing the scores 
collapsed across sex and age groups. 

The overall post-test means from the analysis of covariance adjusted 
for chronological age and PPVT raw score are presented graphically in 
Figure 14.3, 
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10 - 



5- 



0 



TV-HV-MC TV-HV TV Only Control 

13.21 13,30 11,40 8.47 



Figure 14.3 



Overall Means Adjusted for PPVT Post- test Raw Score and 
Chronological Age for APT Part 5 









The ANCOVA summary table for APT Part S is r produced below in Table 

14,8. 



Table 14.8 

Analysis of Covariance Table for APT Part 5 
Post-test Raw Score 



Source 


n 2 


d.f. 


Mean Square 


F 


P 


Trt . 


.07 


3 


270.26 


10.01 


<.005 


Sex 


.04 


1 


161.76 


5.99 


<.025 


Trt . by Sex 


.01 


3 


23.93 


0 , 89 




Covs . 




2 


26946,55 


997,93 




Cov. 1 




1 


15.34 


0,57 




Cov. 2 




1 


1622.-19 


60.08 




Error 


.91 


387 


27.00 







A Dunnett’s post hoc comparison indicated that all three treatment 
groups significantly outscored the control group. No significant differ- 
ences were evident among treatments. This suggests that the television 
program alone was responsible for the differences among means which were 
evident on this measure. 



The sex difference (p<. 025) evident from the analysis of covariance 
indicates that females generally outscored their male counterparts on this 
instrument, with the exception that in the TV-HV-MC group males tended to 
outs core females. 

Analysis of covariance on gain scores indicated that a significant 
treatment effect (p<. 01) occurred on overall learning in this area. The 
groups receiving paraprofessional visitation outscored their counterparts 
in the other groups in terms of gain scores . Thus , an examinati on o f only 
post- test scores tends to obscure the effect of the paraprofessional in 
promoting an increased rate of learning of the objectives sampled by this 
subtest. ■ ' . -• V-‘. 'V. X”? ;/-• ^ V ; ' 



APT Part 6 












^ Paftv 6 of the APT is a 61--item test which was developed during the 
third program year to measure objectives which had heen taught but not 
adequately sampled in other parts of the APT. It is similar in format 
And dorti^nt' tb Part 2; of ^t^ 



ent. to .Part 2 : or the APT. 

v- ■■■ : • --i'.iv ii: jet ; i-i\ ■v.%:r.i..r- ' —-vv- v-i.-'X *■■■■ v *• Vy . ?: r ‘ tr-v- 1 '■■■ . '■ r. ■■ ■ •■■■ t: •- — 

. , 

/:•. : ■' 'i. . ! : / •• ■ ■. : - V.;- : :• V- \ 

'• :• - ■ < ■ X ■■ ~ 

; ' ' ’ • - -fll) - ‘ 

, . -V\V, ..A .•■■■■., -■ - • T . - " X ■ ■ -■ . . V / ~X- ■ : '■ 

. - - ... . ■■ . . . li'ii § - . : . . 







t 



Table 14,9 indicates post-test cell means, N's, and standard deviations 
for Part 6 of the APT as well as total post-test means for each treatment 
group . 



Table 14.9 



Post-test Means, N's, and Standard Deviations, Age by 
Treatment by Sex Cells for APT Part 6 
(Pretest not available) 



Age 


Sex 


TV-HV-MC 


tV-TFv 


TV only 


Control 


3 


M 


x=32 , 42 

N=12 

0=11.54 


x= 31,50 
N= 16 
O- 14.28 


x=19 , 00 
' N= 8 

a- 5.27 


x= 20,82 

N= 17 
0= 8.17 


F 


x- 36 . 38 
N=13 
cr= 8.18 


x= 34,72 
N= 18 

CT= 11,10 


x=21 . 00 

N= 5 
0- 9.32 


3T= 22.21 
N= 19 
o= 10,94 


4 


M 


x=40. 85 

N=20 

0=12.32 


x- 37,68 
N= 25 
Q~ 12,65 


x-39,36 

N=14 

0=11,15 


x= 25.68 
N= 17 
a= 9,88 


F 


x-45 , 58 
N=19 
cj= 8.60 


x= 45,39 
N= 22 
0~ 9.70 


x=24 ,13 
N= 8 
0=15.27 


x= ,28.20 
N= 17 
0= 9.11 


3 ' 


M 


x=50.94 
N-16 
0= 6.70 


x- 47.11 
N* 27 
0= 8.54 


x=35 ,58 
N=1 2 
0=13.69 


x= 33.78 
N= 18 
CT= 8.46 


F 


x=47.53 
N=15 
a= 9.18 


x= 47.59 
N= 22 

, o= 8,65 


x-44.16 

N=19 

0=10.78 


x- 40.75 
N= 15 
o= 7.11 


TOTAL , 


x-42. 87 
N=95 
CT= 9.67 


x= 41.35 
N= 1 30 
o= 10,80 


x=34 . 35 

N=66 

0=11.49 


x= 28.99 
N=105 
o= 9. 10 



The overall post- test means from the analysis of covariance adjusted 
for chronological age and.PPVT raw score for APT Part 6 arepresented 
graphically in Figure, 14'.4>?j‘; -r- : - •> ■ 
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Figure 14,4 



Overall Means Adjusted for PPVT Post-test Raw Score and 
Chronological Age for APT Part 6 



The ANCOVA summary is reproduced below in Table 14,10. 



Table 14,10 



Analysis of Covariance Table for APT Part 6 
Post- test Raw Score 



Source 


n 2 


d.f. 


Mean Square 


F 


P 


Trt, 


.95 


•V 3 


2163.49 


26.74 


<.005 


Sex 


.00 


1 


1173.92 


21.93 


<.005 


Trt, by Sex 


.00 


3 


25.73 . 


0.32 




Covs, 




2 


268251 . 74 


3316.04 




Cov, 1- • 




1 


564.75 - rv • 


V ;V6.98 




Cov, 2 




1 


13415.22 


165.83 


\[\ t ( . 


Error 


.05 


: 387 jv 


•*s ; - : . 80 -90 
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At Dunnett ’ s post hoc analysis indicates, fhatr^e tr catmen t , e f f e c t 
apparent from Table 14 . 10 stems -from contrasts among the groups receiving 
visits from the paraprofessional and those wh i.ch do not, as well as from 
differences between the TV only and control groups . Thus , the television 
program seemed to be providing a basic level of learning for the children 
which was substantially increased by thd paraprofessional. 

. - - j- ; - •- yy ; y -v Y\ ’ v 'Y y.yy\ ■ ' y - ' '/• - ; y .:. "■ , Y' I yyY.;: Y'Y.: '';y;.yy ' . . .;;■■■■ ■ ■; '■ y y y y . 

- 

The sex effect which .was apparent : on this subtest with’ one exception 
favored the females over all ages and treatments .Male 5-year-olds in 
the TV- HV-MC group outs cored their female counterparts here, as they did 
on the remainder of the APT. As was : suggested earlier the van may well 



■ Y ■ ■ Y. . ... , . ...■ ^ . Y Y Y Y .Y:' ■ ■ Y Y:Y Y,y:!-. Y;..Y;YY ; r.Y'YY’Y; : ;YY... Y YyY'.Y V V: ■ , ,.y..y:.y.y y . y : ;.y. ■ Y: 

•• — 




; - • yy-y Y : : Y.; - y .y ’ %-'■ >f,^- ■ ’■ * - n ^ 

. — — — • 



: 






mm 






Pretest scores on Part 6 of the APT are not available, since that part 
of the instrument was developed during the third year of the program. 



Summary 

All subtests of the curriculum specific measure (APT) administered 
during the third year of the program's development indicate significant 
treatment effects for the groups viewing the television program and visited 
by the paraprofessional. The van appeared to have an effect on the child's 
ability to establish rapport with the tester and seemed to be increasing the 
involvement of older boys in the program's objectives. 

Overall, it seems likely that the paraprofessional made the largest con- 
tribution to the objectives measured by the APT, The groups which were ex- 
posed to the home visitor successfully answered more than thirty items (and 
corresponding objectives] over their counterparts in the control group in 
terms of total APT scores. Table 14.11 indicates total APT scores combining 
Parts 2, 5, 6 and the interview. 



Table 14,11 

Sum of APT, Interview, Parts 2, 5 and 6 
TV-HV-MC TV-HV TV Control 



104.97 101.98 93.62 74.37 



An analysis of the data from the third year's evaluation reinforces 
conclusions reached in evaluations of the two previous years— that is, the 
television program provides the "raw material" for the curriculum, while the 
home visitor effectively reinforces the program's cognitive objectives. 

It should be noted that although no significant differences were evident 
for the group visited by the mobile facility, this may be due to the variance 
in attendance on the mobile facility within the group designated as "TV-HV- 
MC. " A t-test iridicatedyth at children, who visited the mobile classroom more 
than 60 percent of the time when it was avail able significantly outscored 
those who attended less than 40 percent of the time on APT Part 6 (x.=44 , 
^2=37, t=2. 40) ; Thus , attendance on' ^lie van does seem to: be related to 
learning of cognitive objectives. - "J- •: vj.:r . V;'„7, - 
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